Skip to content

Conversation

@forrestmurray-db
Copy link
Contributor

This feature is designed to improve facilitation and participation in the discovery stage. The current problems in short:

  • Sparse findings from participants. Current questions allow for low-information responses which make the job of formulating rubric questions much more difficult.
  • High cognitive overload on the facilitator. Facilitators have to work to understand the domain of the agent and the nuances of quality while simultaneously thinking of ways to drive discussion which is very difficult.

The assisted experience:

  • Generates targeted questions for individual participants to spark deep thought.
  • Aggregates and summarizes findings to help the facilitator facilitate.

forrestmurray-db and others added 10 commits January 20, 2026 11:46
Defines the discovery phase facilitation feature:
- Per-trace structured view with 5 category buckets
- Real-time classification of participant findings
- Facilitator-controlled question generation (broadcast per trace)
- Auto-detected disagreements between participants
- Promotion workflow to draft rubric staging area
- Fuzzy progress for participants (no category bias)

Co-Authored-By: Claude <noreply@anthropic.com>
Complete implementation of the Assisted Facilitation v2 architecture across all 6 phases:

Phase 1: Database & Models
- Added 5 new database tables (ClassifiedFinding, Disagreement, TraceDiscoveryQuestion, TraceDiscoveryThreshold, DraftRubricItem)
- Added corresponding Pydantic models with proper configuration
- Updated relationships in WorkshopDB and TraceDB

Phase 2: Classification Service
- Created ClassificationService with real-time finding classification
- Added DSPy signatures for ClassifyFinding and DetectDisagreements
- Implemented local classification heuristic (placeholder for LLM)

Phase 3: Discovery Service Updates
- Added 6 new methods: submit_finding_v2, get_trace_discovery_state, get_fuzzy_progress, promote_finding, update_trace_thresholds
- Implemented fuzzy progress (exploring/good_coverage/complete) for participants
- Added structured discovery state for facilitators

Phase 4: API Endpoints
- Added 7 new REST endpoints for assisted facilitation v2
- Endpoints cover findings submission, discovery state, promotion, and threshold updates

Phase 5: Client Updates
- Created TraceDiscoveryPanel component for facilitator view
- Created DraftRubricPanel component for promoted findings
- Added 5 new React hooks to useWorkshopApi.ts

Phase 6: Testing
- Created test_classification_service.py with 7 tests (all passing)
- Created test_discovery_service_v2.py with 6 tests (all passing)
- 100% test pass rate (13/13 tests)

Key Features:
- Real-time finding classification (5 categories)
- Fuzzy progress indicator for participants
- Disagreement detection framework
- Draft rubric staging area
- Per-trace threshold management

Co-Authored-By: Claude <noreply@anthropic.com>
- Add E2E tests for classification, dashboard, discovery, and rubric promotion
- Update auth test helper to handle workshop selection for facilitators
- Update API mocker to support new workshop scenarios
- Replace migration 0004 with 0008 for randomization columns
- Update workshop router and database service

Co-Authored-By: Claude <noreply@anthropic.com>
Brings in verification and testing improvements including:
- Claude configuration and contributing guidelines
- Enhanced test coverage and test utilities
- Updated E2E tests with better workshop selection handling
- Improved justfile commands
- Updated spec coverage documentation

Co-Authored-By: Claude <noreply@anthropic.com>
- Add .withRealApi() to all TestScenario builders so tests use real database
- Add @SPEC:ASSISTED_FACILITATION_SPEC tag to all 5 test files
- Add ASSISTED_FACILITATION_SPEC to known specs in analyzer and docs
- Fix justfile to export E2E_API_URL for dynamic port support

Co-Authored-By: Claude <noreply@anthropic.com>
Resolve conflicts by merging both feature sets:
- Keep assisted facilitation v2 features (discovery questions, classification)
- Keep JSONPath display customization features
- Merge database columns: discovery_questions_model_name + input/output_jsonpath
- Merge frontend hooks for both feature sets
- Update SPEC_COVERAGE_MAP with both RUBRIC_SPEC and TRACE_DISPLAY_SPEC updates

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant